

is Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



CORRECTED 
NOTICE OF ALLOWANCE AND FEE(S) DUE 



05/16/2006 



EXAMINER 



PHILIPS ELECTRONICS NORTH AMERICA CORPORATION 
CORPORATE INTELLECTUAL PROPERTY 
1000 WEST MAUDE AVE 
SUNNYVALE, CA 94085 



LE,VU 



ART UNIT 



PAPER NUMBER 



2621 

DATE MAILED: 05/16/2006 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/044,876 


10/26/2001 


Chien-Hsin Lin 


US018143 


9838 



TITLE OF INVENTION: VIDEO ARTIFACT IDENTIFICATION AND COUNTING 



APPLN. TYPE 


SMALL ENTITY 


ISSUE FEE 


PUBLICATION FEE 


TOTAL FEE(S) DUE 


DATE DUE 



nonprovisional NO $1400 $300 $1700 08/16/2006 



THE APPLICATION IDENTIFIED ABOVE HAS BEEN EXAMINED AND IS ALLOWED FOR ISSUANCE AS A PATENT. 
PROSECUTION ON THE MERITS IS CLOSED . THIS NOTICE OF ALLOWANCE IS NOT A GRANT OF PATENT RIGHTS. 
THIS APPLICATION IS SUBJECT TO WITHDRAWAL FROM ISSUE AT THE INITIATIVE OF THE OFFICE OR UPON 
PETITION BY THE APPLICANT. SEE 37 CFR 1.313 AND MPEP 1308. 

THE ISSUE FEE AND PUBLICATION FEE (IF REQUIRED) MUST BE PAID WITHIN THREE MONTHS FROM THE 
MAILING DATE OF THIS NOTICE OR THIS APPLICATION SHALL BE REGARDED AS ABANDONED. THIS 
STATUTORY PERIOD CANNOT BE EXTENDED . SEE 35 U.S.C. 151. THE ISSUE FEE DUE INDICATED ABOVE 
REFLECTS A CREDIT FOR ANY PREVIOUSLY PAID ISSUE FEE APPLIED IN THIS APPLICATION. THE PTOL-85B (OR 
AN EQUIVALENT) MUST BE RETURNED WITHIN THIS PERIOD EVEN IF NO FEE IS DUE OR THE APPLICATION WILL 
BE REGARDED AS ABANDONED. 

HOW TO REPLY TO THIS NOTICE: 

I. Review the SMALL ENTITY status shown above. 

If the SMALL ENTITY is shown as YES, verify your current 
SMALL ENTITY status: 

A. If the status is the same, pay the TOTAL FEE(S) DUE shown 
above. 

B. If the status above is to be removed, check box 5b on Part B - 
Fee(s) Transmittal and pay the PUBLICATION FEE (if required) 
and twice the amount of the ISSUE FEE shown above, or 



If the SMALL ENTITY is shown as NO: 

A. Pay TOTAL FEE(S) DUE shown above, or 

B. If applicant claimed SMALL ENTITY status before, or is now 
claiming SMALL ENTITY status, check box 5a on Part B - Fee(s) 
Transmittal and pay the PUBLICATION FEE (if required) and 1/2 
the ISSUE FEE shown above. 




II. PART B - FEE(S) TRANSMITTAL should be completed and returned to the United States Patent and Trademark Office (USPTO) with 
your ISSUE FEE and PUBLICATION FEE (if required). Even if the fee(s) have already been paid, Part B - Fee(s) Transmittal should be 
completed and returned. If you are charging the fee(s) to your deposit account, section "4b" of Part B - Fee(s) Transmittal should be 
completed and an extra copy of the form should be submitted. 

III. All communications regarding this application must give the application number. Please direct all communications prior to issuance to 
Mail Stop ISSUE FEE unless advised to the contrary. 

IMPORTANT REMINDER: Utility patents issuing on applications filed on or after Dec. 12, 1980 may require payment of 
maintenance fees. It is patentee's responsibility to ensure timely payment of maintenance fees when due. 

Page 1 of 3 

PTOL-85 (Rev. 01/06) Approved for use through 04/30/2007. 



PART B - FEE(S) TRANSMITTAL 
Complete and send this form, together with applicable fee(s), to: Mail 



Mail Stop ISSUE FEE 
Commissioner for Patents 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
or Fax (571)-273-2885 



INSTRUCTIONS: This form should be used for transmitting the ISSUE FEE and PUBLICATION FEE (if required). Blocks 1 through 5 should be completed where 
appropriate. All further correspondence including the Patent, advance orders and notification of maintenance fees will be mailed to the current correspondence address as 
indicated unless corrected below or directed otherwise in Block 1, by (a) specifying a new correspondence address; and/or (b) indicating a separate "FEE ADDRESS" for 

maintenance fee notifications. 

current correspondence address (Note: Use Block i for any change of address) — — Note: A certificate of mailing can only be used for domestic mailings of the 

Fee(s) Transmittal. This certificate cannot be used for any other accompanying 

Eapers. Each additional paper, such as an assignment or formal drawing, must 
ave its own certificate of mailing or transmission. 

7590 05/1672006 

Certificate of Mailing or Transmission 

PHILIPS ELECTRONICS NORTH AMERICA CORPORATION I hereby certify that this Fee(s) Transmittal is being deposited with the United 
CORPORATE INTELLECTUAL PROPERTY States Postal Service with sufficient postage for first class mail in an envelope 

1000 WEST MAUDE AVE 



addressed to the Mail Stop ISSUE FEE address above, or being facsimile 
transmitted to the USPTO (571) 273-2885, on the date indicated below. 



SUNNYVALE, CA 94085 



(Depositor's name) 



(Signature) 



(Date) 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/044,876 


10/26/2001 


Chien-Hsin Lin 


US018143 


9838 



TITLE OF INVENTION: VIDEO ARTIFACT IDENTIFICATION AND COUNTING 



APPLN. TYPE 


SMALL ENTITY 


ISSUE FEE 


PUBLICATION FEE 


TOTAL FEE(S) DUE 


DATE DUE 


nonprovisional 


NO 


$1400 


$300 


$1700 


08/16/2006 


EXAMINER 


ART UNIT 


CLASS-SUBCLASS 






LE,VU 


2621 


375-240290 







1 . Change of correspondence address or indication of "Fee Address" (37 
CFR1.J63). 

CD Change of correspondence address (or Change of Correspondence 
Address form PTO/SB/122) attached. 

Q "Fee Address" indication (or "Fee Address" Indication form 
PTO/SB/47; Rev 03-02 or more recent) attached. Use of a Customer 
Number is required. 



2. For printing on the patent front page, list 

(1) the names of up to 3 registered patent attorneys 
or agents OR, alternatively, 

(2) the name of a single firm (having as a member a 
registered attorney or agent) and the names of up to 
2 registered patent attorneys or agents. If no name is 
listed, no name will be printed. 



3. ASSIGNEE NAME AND RESIDENCE DATA TO BE PRINTED ON THE PATENT (print or type) 

PLEASE NOTE: Unless an assignee is identified below, no assignee data will appear on the patent. If an assignee is identified below, the document has been filed for 
recordation as set forth in 37 CFR 3. 1 1 . Completion of this form is NOT a substitute for filing an assignment. 

(A) NAME OF ASSIGNEE (B) RESIDENCE: (CITY and STATE OR COUNTRY) 



Please check the appropriate assignee category or categories (will not be printed on the patent) : Ql Individual Q Corporation or other private group entity Q Government 



4a. The following fee(s) are enclosed: 
Q Issue Fee 

O Publication Fee (No small entity discount permitted) 
Q Advance Order - # of Copies 



4b. Payment of Fee(s): 

Q A check in the amount of the fee(s) is enclosed. 

D Payment by credit card. Form PTO-2038 is attached. 

□ The Director is hereby authorized by charge the required fee(s), or credit any overpayment, to 
Deposit Account Number (enclose an extra copy of this form). 



5. Change in Entity Status (from status indicated above) 

□ a. Applicant claims SMALL ENTITY status. See 37 CFR 1.27. 



□ b. Applicant is no longer claiming SMALL ENTITY status. See 37 CFR 1.27(g)(2). 



The Director of the USPTO is requested to apply the Issue Fee and Publication Fee (if any) or to re-apply any previously paid issue fee to the application identified above. 
NOTE: The Issue Fee and Publication Fee (if required) will not be accepted from anyone other than the applicant; a registered attorney or agent; or the assignee or other party in 
interest as shown by the records of the United States Patent and Trademark Office. 



Authorized Signature _ 



Typed or printed name _ 



Date 



Registration No. 



This collection of information is required by 37 CFR 1.31 1. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to process) 
an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, including gathering, preparing, and 
submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete 
this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. 
Box 1450, Alexandria, Virginia 223 13-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, 
Alexandria, Virginia 2231 J- 1450. 

Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. 



PTOL-85 (Rev. 01/06) Approved for use through 04/30/2007. 



OMB 065 1 -0033 U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE 



United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



10/044,876 



10/26/2001 



Chien-Hsin Lin 



7590 



05/16/2006 



PHILIPS ELECTRONICS NORTH AMERICA CORPORATION 
CORPORATE INTELLECTUAL PROPERTY 
1000 WEST MAUDE AVE 
SUNNYVALE, CA 94085 



US018143 



9838 



EXAMINER 



LE.VU 



ART UNIT 



PAPER NUMBER 



2621 

DATE MAILED: 05/16/2006 



Determination of Patent Term Adjustment under 35 U.S.C. 154 (b) 

(application filed on or after May 29, 2000) 

The Patent Term Adjustment to date is 762 day(s). If the issue fee is paid on the date that is three months after the 
mailing date of this notice and the patent issues on the Tuesday before the date that is 28 weeks (six and a half 
months) after the mailing date of this notice, the Patent Term Adjustment will be 762 day(s). 

If a Continued Prosecution Application (CPA) was filed in the above-identified application, the filing date that 
determines Patent Term Adjustment is the filing date of the most recent CPA. 

Applicant will be able to obtain more detailed information by accessing the Patent Application Information Retrieval 
(PAIR) WEB site (http://pair.uspto.gov). 

Any questions regarding the Patent Term Extension or Adjustment determination should be directed to the Office of 
Patent Legal Administration at (571)-272-7702. Questions relating to issue and publication fee payments should be 
directed to the Customer Service Center of the Office of Patent Publication at l-(888)-786-0101 or 
(571)-272-4200. 



PTOL-85 (Rev. 01/06) Approved for use through 04/30/2007. 



Page 3 of 3 



me 



A 1 


Application No. 


Appticant(s) | 


Notice of Allowability 


10/044.876 


LINETAL 


Examiner 


Ait Unit 






Vu Le 


2613 





. - The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 
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THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 
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Allowable Subject Matter 

1 . Claims 1-14 are allowed. 

2. The following is an examiner's statement of reasons for allowance: 

The prior art (WO 01/20912) closest to the present invention belongs to the same 
assignee. The prior art and the present invention utilize similar algorithm for identifying 
block artifacts in the digital video pictures. However, Examiner fails to ascertain from 
the prior art disclosure or suggestion the limitations of "...using first and second loops 
for each of horizontal and vertical count table entries, the first loop being adapted to 
increase the count table entries up to the length of a remainder value of a detected 
artifact value and the second loop being adapted to increase the count table entries 
using a quotient value of a detected artifact value" as recited in claim 1. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Contact 

3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Vu Le whose telephone number is 703-308-6613. The 
examiner can normally be reached on M-F 8:30-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris Kelley can be reached on 703-305-4856. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Description 

The present invention relates digital image coding. More precisely, the invention relates to a post-processing 
method for reducing artifacts in block-coded digital images, and to a post-processing device suitable for actuating such 
a method. 

With the cfiffusion of digital communication systems, digital images are more and more used. This has led to the 
diffusion of etiQ and vkteo cameras with cfiQrtal acquisition and processing capability. 

In order to better exploit storage means and transmission bandwidth, digital image compression standards have 
been developed, such as JPEG for still images, and MPEG-1 and MPEG-2 for digital television image sequences. 

The above-referred compression standards provides for block-coding based on Discrete Cosine Transform (OCT). 
A digital image is divided into blocks of pixels, and each block is encoded independently from the others. DCT coeffi- 
cients for the pixels of each block are evaluated and a quantization matrix is applied to the OCT coefficients to reduce 
the information to be stored or transmitted. When the image is to be displayed, it must be decoded in advance. 

Due to the quantization process, these image compression methods are lossy, i.e. they cause a loss of information 
in the decoded image with respect to the original image. The decoded image can thus present noticeable degradation, 
mainly consisting of two kinds of artifacts known in the art under the names of "grid noise" and "staircase noise". 

In order to reduce the image degradation, post-processing methods of the decoded image have been proposed 
which allcw for attenuating grid noise and staircase noise. 

In view of the state of the art described, it is an object of the present invention to provide a new post-processing 
method for reducing artifacts in block-coded digital images. 

According to the present invention, such object is attained by means of a post-processing method for reducing arti- 
facts in block-coded digital images, characterized by providing for: 

a) dividing an input image into a plurality of image Uocte; 

b) for each image block, estimating global features of said image Mock provicfing information on an average content 
of image edges along the horizontal and vertical directions of said image block; 

c) for each pixel of an image block under examination, estimating local features for said pixel providing information 
on the content of image edges along the horizontal and vertical directions of an image area around said pixel: 

d) modifying the value of said pixel according to both said global features of the Image block to which said pixel 
belongs and said local features of the image area around said pixel. 

Also according to the invention, there is also provided a post-processing device for actuating said method, charac- 
terized by comprising: 

• first means supplied with an input image for est i mating global features of an image block under examination, said 
global features providing information on an average content ol image edges along the horizontal and vertical direc- 
tions of said image block; 

• seoond means suppQed with said input image for estimating focal features for each pixel of the image block under 
examination, said local features provfclng informatfon on the content of image edges along the horizontal and ver- 
tical directions of an [mage area around said pixel; 

- third means suppGedwift said global features arid said local features for modrryi 
to both said global features and said local features. 

Tto features and advantages of the p 
of an embodiment thereof, illustrated as a non-limiting example in the annexed drawings, who-ein: 

Figure 1 is a schematic block diagram illustrating the principle of operation of a method according to the present 
invention; 

Figure 2 shows a digital image divided otto image blocks; 

Figure 3 shows in detail an image block into which the digital image of Figure 2 is divided; 
Figure 4 shows an array of pixels of the image block of Figure 3 ; 

Figure 5 shows an image sub-block of the image block of Figure 3 used tor evaluating global features of the image 
block; 

Figure 6 shows an horizontal processing window used for evaluating local features in the horizontal direction for a 
generic pixel of the image block; 

Figure 7 shows a vertical processing window used for evaluating focal features in the vertical direction for said 
generic pixel; 

Figures 8 and 9 shews two membership functions used to perform a fuzzy computation : 
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Figure 10 is a block diagram of a device seconding to toe present invention; 
Figure 11 shorn the structure of two Hocks of the device of Figure 10; and 
Figure 12 is a block diagram of other two Hods of the device of Figure 10. 

With reference to Rgure 1 . there is shown a Hock diagram illustrating the principle of operation of the post-process- 
ing method according to the present invention. An input decoded compressed dtgrial image In is suppfied to a Feature 
Extraction Hock FE. Block FE provides for analyzing the image to evaluate global and local features tfiereof. The global 
and local features* respectively QF and IF. of the image In evaluated by block FE are supplied as inputs to a Fuzzy 
Process Hock FUZZY which, according to fuzzy rules, determines parameters FA suitable for determining the Kind of 
filtering to be performed, in accordance to the global and local features GF and FL of the image. The parameters FA 
calculated by block FUZZY are suppfied to a Filter Compositiori Hock FC which according to said parameters FA deter- 
mines the type of filtering to performed out of a set of predefined fitters (Hock FS). Filter parameters FP determined by 
Hock FC are then supplied to a Processing Hock PROG also supplied directly with the input image In, which performs 
the filtering of the input image In according to the filter parameters FP to provide a po6t-processed output image Out. 

It appears that tfie kind of filtering to be performed on the decoded input image m is chosen after an estimation of 
the global and local features of thi decoded input image. For inrage areas rmargrkJ noise arri near an edge, alow-pass 
filtering is performed, to reduce both staircase noise and grid noise. For areas containing fine details (image edges and 
texture), no filtering is performed. Thus, the method according to the present invention provides for performing a non- 
linear adaptive filtering on the pixeJs of the decoded image. 

The principle of operation outlined above wffl be now described in detai. 

As shown in Figure 2. the input image In is partitioned Into image Hocks IB, each containing an equal number of 
pixels. A typical dimension of the Hocks is 8*8 pixels (Figure 3). but this is not however to be intended as a limitation, 
since other Hock cfimensions are suHabfa 

The image Hocks IB of the input image in are scanned Cne by fine starting from this top-left block to the bottom-right 
one For each image Hock IB, the Feature Extraction Hock FE in Figure 1 determines the global and local features GF 
and IF. 

Global features of the image Hock IB under examination are determined by applying horizontal and vertical Sobe! 
operators: 

El h12 h13i 
1h22h33; 
1 h32 h33j 

rv11 vl2 V13-I 

vertical Sobel operator (Vsob) : v21 v22v23 

Lv31 v32 v33J 

to each pixel belonging to an image 6uMriock internal to the image Hock IB. For example, the following Sobel opera- 
tors: 




are applied to a 6*6 pixel image sub-block ISB (gray area in Rgure 5). As shown in Figure 4, for each pixel Pi of the 
image sub-Hock ISB a 3*3 array of rraghboring pbcBfc M centered in pixel Pi is considered, and the values of the pixels 
of said array M are multiplied by the coefficients of the horizontal and vertical Sobel operators, to obtain: 



Hsob - (P3+P8+2T5) - (P1*P6+2*P4). 
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Vaob u (P6*P8+2'P7) - (P1+P3+2*P2). 

where P1 -P4 and P5-P8 are the values of the pixels (gray levels). 

As known, horizontal and vertical Sobel operators perform a filtering capable of detecting edges along the horizon* 
tal and vertical direction, respectively. 

The output values of the horizontal Sobel operators calculated for the pixels of image sub-block ISB are accumu- 
lated to obtain an accumulated value Aoc(Hsob), and the output values ol the vertical Sobel operators calculated for the 
pixels of image sub-block ISB are accumulated to obtain an accumulated value Acc(Vsob). AccfHsob) gives the high- 
frequency content in the horizontal direction (vertical edges) of the image block IB. Aoc(Vsob) gives the high-frequency 
content in the vertical direction (horizontal edges) of image block IB. Thus, Aoc(Hsob) and Acc(Vscb) respectively pro- 
vide the degree of "edgeness" of the image block under examination in the vertical and horizontal directions, ft is to be 
noted that in order to evaluate the global features GF of the image block IB under examination, only the pixels belonging 
to this block are considered (by applying 3*3 Sobel operators to the 6*6 image sub-block ISB, it is not necessary to take 
into consideration pixels belonging to neighboring image blocks). 

Global features GF of the image block under examination can be formed by the accumulated values Acc(Ksob) and 
Acc(Vsob). Alternatively, the global features GF of the image block can be formed by an average of the accumulated 
values Acc(Hsob) and AccO/sob), to obtain the average number of edges in the rwrizontal and vertical directions in th8 
image block under examination. 

local features LF of the image block IB are estimated acoorcfing to the following method. All the pixels of the image 
block IB under examination are scanned fine by fine starting from the to^ 

pixel a horizontal processing window is applied: a preserved number of pixels respectively preceding and following the 
pixel under consideration and belonging to the same Image fine of the pixel under consideration Is considered. A suita- 
ble horizontal processing window HPW is shown in Figure 6, which is a horizontal 1 "5 processing window: for a given 
pixel, the two preceding pixels Pa Pb a 

Figure 6 there is shown by way of example the horizontal processing window HPW associated to the f irst pixel Px of the 
image blrxk. ft shoukl be noted that 

pixels belonging to neighboring image blocks: this is for example the case of the first second, seventh and eight pixel 
of each One of pixels of the image block IB under examination. 

The horizontai Sobel operator Hsob previously mentioned is applied to each pixel Pa, Pb, Px. Pc, Pd in the hcrizorv 
taJ processing window HPW, to obtain five output values HS1-HSS. values HS14JS5 provide the local features in the 
horizontal direction tor the pixel under examination Px, i.e. the high-frequency content in tire horizontal (fraction of the 
image region around the pixel under examination. 

Similarly, a vertical processing window is applied to each pixel of the image block IB. The vertical processing win- 
dow is formed by the pixel under consideration Px, and a prescrtoed number of pixels belonging to the same column as 
and preceding and foflowing the pixel under consideration; for example, as 6hown in Figure 7 the vertical processing 
window VPW can have dimensions identical to the horizontal processing window HPW (5*1), and thus contains two pix- 
els Pe, Pf preceding pixel Px and two pixels Pg. Ph following pixel Px in the vertical rjrecfort 

The vertical Sobel operator Vsob deviously mentioned is then applied to each pixel of the vertical processing win- 
dow VPW to obtain five output values VS1 -VS5. Values VS1 -VS5 form the local features in the vertical direction for the 
pixel under examination, i.e. the high-frequency content in the vertical direction of an image region around the pixel 
under examination. 

The global features GF tor the image block IB under examination (te.. the two accumulated values AccfHsob) and 
Acc(V6ob) or, in alternative, the average value of Acc(Hsob) and AccfVscb)) and the local features LF for the pixel 
under examination inside said image block (the ten values HS1-HS5 and VS1-VS5) are then suppGed to the Fuzzy 
Process block FUZZY. This block provides for evaluating the degree of membership of a generic value HSi and Vsi 
0-1 ..9 to two fuzzy sets "Small* and "Bier; these degree of marrtoership can be evaluated by applying to HSi. VSi the 
membership functions depicted in figures 8 and Synthase fibres, 

features OF of the Image block under examination, La on the accumulated values Aoc(Hsob) and Acc(Vsob) or on the 
average of the accumulated values; in the first case, Th1 and Th2 are different for the Hsi and Vsi values; in the second 
case, Thi and Th2 are the same tor Hsi and Vsi values. 

Fuzzy rules having as antecedents the degrees of membership of the output values HSi and VSi to the two fuzzy 
sets "Small" and W are then evaluated. This means thm 32 rules are fo be evaluated for both t^ 
tical directions. However, afl those fuzzy rides having the same consequence are synthesized in one rule only by an else 
operator. In this way, the system complexity is reduced, and a total of nine rules for each direction have to be evaluated. 

The following fuzzy rules are applied to the five values HS1-HS5 associated to the horizontal direction : 

1. ffWSl & Small andH§2 is Small andHSS b Small andHSA is Small andHSS is Small, then a1 & Big; 

2. (f HSI is Small and HS2 is Small and HS3 is Small and HS4 is Small andHSS is Big, then o2 is Big; 
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3. tfH$^ is Small and HS2 is Small and HS3 is Small and HS4 is Big and MS5 ts Small, then a3 es Big; 

4. //HS1 is Small and HS2 is SmaO and HS3 is Small and HS4 is Big and HS5 fe Big, then a4 is Big; 

5. J/HS1 & Small and HS2 is Big and HS3 & Small and HS4 is Small and HS5 is Small, t/wn a5 cs Big; 

6. //HS1 is Big and HS2 is Small and HS3 is Small and HS4 is Small and HS5 is Small, then o6 is Big; 

7. ffHS1 is Big andHS2 is Small andHSQ is Small andHS4 is Small and HS5 is Big. then a7 is Big; 

8. HS1 is Big and HS2 is Big and HS3 is SmaO and HS4 is Small and HS5 is Smal. fnen a8 is Big. 

The activation level of each rule depends on the degree of memberships of the pattern of output values HSiof the 
horizontal Sobel operator applied to the five pixels of the horizontal processing window HPW; the degree of member- 
ships depend in turn on the global features GFofthe image block to which the pixel under examination belongs, The 
activation level of the else (ninth) rule is computed as aefse = (1 - aave) , where oave is the average activation degree 
of fuzzy rules 1 to 8. a1 toa8 and aelse, and a similar set of nine activation degrees for the fuzzy rules applied to values 
VS1 -VS5) form the oirtput FA of the fuzzy process block FUZZY in Figure 1 . 

Each one of the above-listed rules is associated to a respective set of predef ined filter parameters, which are stored 
as a look-up table in block FS of Figure 1 . Suitable predefined filter parameter sets are for example: 

Rule 1: (d 1=1.0, d 2-1.0, c13»1.0, c14=1.0, c15=1.0) if the pixel under examination Bes outside the image sub- 
biock ISB, and (c11=0.0. c12=1.0. c13«1 A c14-1 .0. d 5=0.0) if the pixel under examination lies inside the image 
subUocklSB; 

Ride 2: (C21.0.5. C22-1.0. 023=1.0. C24-1.0, C25-0.0); 
Ride 3: (C31-0.5. C32-1.0. C33-1 .0. C34-0.0, c35=0.0); 
Rule 4: (c41=05. C42-1.0. C43-1.0. C44-0.0, C45-0.0); 
Rule 5: (c51=0.0, c52=0.0, C53-1.0. c54=1.0, c55*0.5); 
Rule 6: (c61=0X), c62=1.0, c€3-1 .0, c64=1.0, c65=0.5); 
Rule 7: (C71-0.0, C72-1.0. c73«1.0, C74-1.0, C75-0.0); 
Rule 8: (C81«0A c82«0.0, c83=1.0, CS4-1.0, C85-0.5); 
Else rule: (c9 1-0.0. C92-0.0. c93«1X>. c94=0.0, c95=0.0). 

The parameters FP of the filter to be applied to the five pixels of the horizontal processing window HPW are calcu- 
lated as a weighted average of the nine filters described above, with weight factors formed by the activation degrees a1 
to a8 and aetee of the respective fuzzy rules. 

Assuming that ai is the activation degree of the i-th fuzzy rule (U1..9). the ninth fuzzy rule being the else fuzzy rule 
(a9 - aelse ). and cfj are the coefficients of the Mh tater r>i ..9. j=i ..5). the weight factor appfied to the Mh filter, asso- 
ciated to the Hh fuzzy rule ts: 

Fi » ai • dj 

andmecoeffkxerrtsHjofthef^ 
are given by: 



Soi-cij 
n N 

where N is a normalization factor. 

The horizontally-filtered value Ex of the pixel Px under examination (at the center of the horizontal processing win- 
dow) ts then calculated as a weighted average of the values of the pixels Pa. Pb. Px, Pc and Pd belonging to the hori- 
zontal processing window HPW, with weight factors formed by the coefficients Hj: 

Px o HfPa* H2*Pb+ H3*Px+ H4'Pc+ H5*Pd. 

Similar calculations are performed for the vertical <fir action, starling from the output values VS1 -VS5 of the vertical 
Sobel operators appfied to the pixels Pe^ Pf t Px, Pg aid 

(jal ..5) of the f ater for the vertical direction are calculated in a way completely simiar to that used for determining the 
coefficients Hj: 
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VI- "tT- 

5 

where pi (i= V.9) are the activation degrees of nine fuzzy rules for the vertical (fraction (similar to those fisted above for 
the horizontal direction) and qj (j-1 . .5) now are the predefined fitter parameters associated to the Mh fuzzy rule for the 
vertical direction. The coefficients VJ are then applied to the pixels In the vertical processing window VPW to calculate 
a weighted average of the same. The filtered value of the pixel Px under examination, tittered in both the horizontal and 
10 vertical direction, is provided at the output Out of the processing, block PROG 
The value of the pixel Px under examination to be muHipM 
obtained after having applied to the pixels in the horizontal processing window HPW the horizontal filter Hj r>i ..5): 

Out - V1 *Pe ♦ V2*Pf + V3*Px ♦ V4*Pg + VS'Ph. 

Alternatively. It is possible to evaluate first the vertically-filtered value Pxof the pixel under examination: 

Px - V1 *Pe ♦ V2*Pf ♦ V3*Px ♦ V4*Pg + VS*PK 

go and then performing the tittering in the horizontal direction applying to this value the respective coefficient H3 of the hor- 
izontal filter Hj: 

Out = H1 *Pa + HT Pb + H3*Px + H4*Pc ♦ H5*Pd. 

25- The sequence is of no importance, the irnportant thing to be underlined being that at the end of the process the value 
of the pixel under examination is the result of both an horizontal and a vertical fftertng. 

Figure 10 is a block diagram off a device suitable for actuating the method previously described. The device com- 
prises two main blocks: a first block 1 evaluates the global features OF of the Image blocks IB the Image to be post- 
processed is divided in, and a second block 2 evaluates the local features LF of the pixels of the image and performs 
so the filtering accorcSng to both the global features and the local features. 
It is assumed that the image to be post-processed is scajinedliiwb^ 
of pixels of the input image scanned line by Gna Btock1 is supplied with signal In; signal In also supplies a cascade of 
two line memories LM1 and LM2 whose outputs states block 1. 

Inside block 1. signal In and the otrtputs erf line memories LM1 and LM2 supply a block 3 of pixel delays suitable for 
35 irrplernerrtng a 3*3 pixel window which is used to calculate horizontal and vertical SobeJ operators for the pixels of the 
6 # 6 image sub-block ISB inside each image block IB. Bkx* 3 supplies a btock 4 whk* calculates the outputs Hsoband 
Vsob of the horizontal and vertical SobeJ opentiorc for those pixete of the cum 

sub-blocks ISB of each image block IB. The outputs Hsob and Vsob of block 4 are suppfied to an accumulator block 5 
wherein they are accumulated. After eight image Ones, i.e. a One of image blocks IB, have been scanned, theaocumu- 
40 lated values Acc(Hsob). AccfVsob) (or alternatively the average thereof) for each image block IB are stored in a mem- 
ory block 6. 

The output of line memory LM2 supplies a cascade of eight further line memories LM3-lMia Block 2 is supplied 
in parallel with the outputs of line memories UM4-LM10. In this way, evaluation of the local features and calculation of 
the fSter parameters starts after block 1 has estimated the global features GF for a fine of image blocks IB. 
45 Inside block 2, a block 7 of pixel delays is suppfied with the outputs of fine memories LM4-LM1 0; by means of the 
Gne memories LM4-LM10 and the pixel delay block 7 it is possarie to implement the 5*1 vertical processing window 
VPW. The outputs L4-L10 of the pixel delays block 7 supply a block 8 which applies ^ 

pixel insidi the vertical processing window VPW. To avoid the use of further line memories, a parallel approach te pre- 
ferred rxcvidng for calculating five vertical Sobel operators in parallel; the outputs of the five vertical Sobel operators 

» VS1-VS5 are supplied to a vertical fuzzy filter 9, which is also supp&ed with the outputs UHJOtf the pixel delay block 
7 and the output MOUT of the memory MEM of block 1. MOUT supplies the global features GF of the image block IB 
currently processed by block 2. i.e. the accumulated value AccfVsob) or. alternatively, the average of Acc(Vsob) and 
Acc(Hsob). The vertical fuzzy fiter block 9 evaluates the degree of membership of values VS1-VS5 to the fuzzy sets 
"Small" and "Big" taking into account the global features provided by MOUT, evaluates the activation levels of the nine 

55 fuzzy rules for the vertical direction, c^^atesthe cx^icients Vj (>1..5) of the vertical filter and appfies the vertical filter 
coefficients Vj to the five pixels Pe, Pf, Px, Pg, Ph in the vertical processing window VPW, to calculate the verticalfy- 
fitteredvsJueJ^olthepfcelta 

9 forms the verticairy-fBtered value Ed of pixel Pd in the horizontal processing window HPW shown in Figure 6 and sup- 
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pUescfirectJy a horizontal fuzzy lifter btodt 10; the output Eg of block 9 also supplies a cascade of four pixel delays D 
whose outputs respectively form the vertical ry-filtered values EkEigkEaof tfie pixels Pc, Px, Pb, Pa in the horizontal 
processing window HPW and 6upply the horizontal fuzzy fmer block 1 0. 

In parallel to blocks 8 and 9. the outputs L7-L9 of the pixel delay block 7 supply a block 11 which applies the hori- 
zontal SobeJ operators to the pixels inside the horizontal processing window HPW. Differently from the vertical sobel 
operators, only one horizontal sobel operator is cafcuteted at a time; a corrpensationdelay block 12 introduces a delay 
for compensating the processing delay of the vertical fuzzy filter block 9. The output of block 1 2. forming the output of 
the horizontal SobeJ operator HS5 applied to pixel Pd of the horizontal processing window In Figure 6. supplies the hor- 
izontal fuzzy fflter block 10 and a cascade of four pixel delays D. the outputs thereof forming the values HS4, HS3, HS2 
and HSl and suppling the horizontal fuzzy filter block 10. The horizontal fuzzy fOter block 10, which is also supplied by 
the output MOITT of the memory block MEM in block 1 providing the value Acc(Hsob) (or alternatively the average of 
values Acc(Hsob) and AccfVsob)), evaluates the degree of membershp of values HS1-HS5 to the fuzzy sets "SmaO" 
and "Big" according to the value of the global features GF provided by MOUT. evaluates the activation levels of the nine 
fuzzy ruJ es described above for the tillering in the horizontal direction, calculates th e coefficients Hi of the horizontal fa- 
ter and applies the parameters Hj to the vertodly-f Itered values f& Ex, E£> £cj of the pixels Pa. Pb, Px, Pc, Pd in 
the horizontal processing window HPW to obtain the horizontaly- and verticalyfittered value Out of the pixel Px under 
examination. 

A control circuit CTRL controls the operation of blocks 1 . 2 and the line memories LM1-LM10. 

Figure 11 shows the structure of the vertical and horizontal Sobel operator blocks 8 and 1 1 of Figure 10. They are 
composed in a straigrttforward way by adders. 

Figure 12 shows the structure of both the vertical fuzzy filter block 9 and the horizontal fuzzy filter Week 10. X1-X5 
are the vertical or. respectively, horizontal Sobel operator outputs VS1-VS5 and HS1-HS5. X1-X5 are supplied to a 
fuzzy rule evaluation block 13 which evaluates the activation degrees 01-09 of the nine fuzzy rules for the vertical direc- 
tion or. respectively, the activation degrees a1-o9 of the nine fuzzy rules for the horizontal direction. The activation 
degrees evaluated by block 13 are supplied to a look-up table of respective predefined fflter parameters F1-F9 (forrning 
block FS in Figure 1). and the outputs of the look-up table, i.a the predefined filter parameters cq multiplied by the acti- 
vation degree of the respective fuzzy rule, are supplied to a filter cc<nposrtion block 1 4 which calculates lhe coefficients 
V1-V5 or, respectively, H1-H5, of the vertical or, respectively, horizontal filter. Said coefficients are then supplied to a 
processing block 15 which is ato supplied with the pbedvato 

ure 10): block 15 applies the filler coefficients to fte pixel values to obtain the filtered value of the pixel under examina- 
tion Px 

Claims 

1. Part-processing method for reckicing artifacts m biock-coded digital images, crwacterized by providing for: 

a) dividing an input image (ln)irnoariuraayof Imagebkxks(B); 

b) for each image block (IB), estimating global features (OF) of said image block providing information on an 
average content of image edges along the horizontal and vertical ejections of said image block (IB); 

c) for each pixel (Px) of an image block (IB) under examination, estimating local features (U^ tor said pixel (Px) 
providing information on the content of image edges along the horizontal and vertical directions of an image 
area near said pixel (Px); / 

d) modifying the value of said pixel (Px) according to both said global features (GF) of the image bkxk (IB) to 
which said pixel (Px) belongs and said local features (LF) of the image area near said pixel (Px). 

2. Method according to daim 1 , characterized (n that step d) provides for: 

d1) defining a set of predefined local features; 

d2) determining degrees of coincidence (a 1 ^a9.p 1 -09)of said local features (LF) of the image area around said 
pixel (Px) with each predefined local features of said set. said degrees of coincidence (a1-a9,p1-p9) depend- 
ing on said global features (OF) of the image block (IB) to which said pixel (Px) belongs; 
d3) making the value of said pixel (Px) equal to a weighted average (Px.Px) of the value of said pixel (Px) and 
of the values of neighboring pixels (Pa.Pd.rVPf), with weight factors (H1-H5.V1-V5) depending on said 
degrees of commence of said local features (LF) with each of said predefined local features. 

& Method according to daim 2, characterized in that said determining the degrees of coincidence (al -09,01-09) in 
step d2) provides for performing a fuzzy calculation. 
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4. Method according to daim 3. characterized in that each of said predefined local features is associated to a respec- 
tive group of predefined weight factors (tij), and each of said weight factors (H1-H5.V1-V5) is calculated as a 
weighted average of corresponding predefined weight factors (dj) of said groups with weight coefficients (<x1- 
a9.pi -p9) depending on said degrees of coincidence of said Coca) features (LF) with each of said predefined local 
features. 

5. Method according to daim 4, characterized in that said estimating global features (GF) of the image block (IB) 
under examination provides for applying horizontal and vertical Sobel operators (Hsob. Vsob) to pixels belonging to 
an image sub-block (ISB) internal to said image block (IB) under examination. 

& Method according to daim 5. characterized in that said estimating global features (GF) of the image block (IB) 
under examination provides for adding outputs of the horizontal Sobel operators (Hsob) applied to each pixel of 
said image sub-block (ISB) to obtain an accumulated output of horizontal Sobel operators (Acc(Hsob)) ( and adding 
outputs of the vertical Sobel operators (Vsob) applied to each rixel of saM 
mutated output of vertical Sobel operators (Acc(Vsob)). 

7. Method according to daim 6, ctau^erized in that sakf c^c^ features (GF) of the image block (IB) under exami- 
nation are formed by said accumulated outputs of the horizontal and vertical Sobel operators 
(Aoc(Hsob).Ace(Vsob)). 

8. Method according to daim 6, characterized in that said global features (GF) of the image block (IB) under exami- 
nation are formed by an average of said accumulated outputs of the horizontal and vertical Sobel operators 
(Acc(Hsob).Acc(Vsob))- 

9. Method according to claim 7 or 8, characterized in that said estimating local features (LF) for a pixel (Px) of the 
image block (IB) under examination provides for: 

d) considering an horizontal processing window (HPW) containing the pixel (Px) under examination and 
neighboring pixels (Pa t Pb,Pc,Pd) belonging to a same image One as the pixel (Px) and preceding and following 
the pixel (Px): 

c2) applying said horizontal Sobel operator (Hsob) to each pixel (Pa-Pd.Px) of the horizontal processing win- 
dow (HPW) to obtain a horizontal pattern of horizontal Sobel operator outputs (HSl -HS5); 
c3) corisidenng a vertical processing window (VPW) containing the pixel (Px) under examination and neighbor- 
ing pixels (Pe.Pf.Pg, Pti) belonging to a same column of pixels as the pixel (Px) and preceding and following the 
pixel (Px): 

<*) applying said vertical Sobel operator (Vsob) to each pixel (Pe-Ph.Px) of the vertical processing window 
(VPW) to obtain a vertical pattern of vertical Sobel operator outputs (VS1-VS5). 

10. Method according to daim 9. characterized in that said horizontal and vertical processing windows (HPW.VPW) 
corrtains each one five pixels and are centered at said pixel (Px) under exarrtrtatton. 

11. Method according to daim 10. characterized in that step d2) provides for determining degrees of membership of 
each horizontal Sobel operator outpirf 

fuzzy set evaluating activa^ 

one predefined horizontal pattern of horizontal Sobel operator outputs, determining degrees of membership of 
each vertical Sobd operator output 

fuzzy set "Kg" and evaluating activation degrees (p1 -09) of a second set of fuzzy rules each one associated with 
at least one predefined vertical pattern of vertical Sobel operator outputs. 

12* Method according to daim 11, characterized in that said determining degrees of membership of the horizontal 
Sobel operator outputs (HS1-HS5) of the horizontal pattern to said first fuzzy sets "Small" and "Big" provides tor 
determining a first and a second membership functions deperxfing on said global features (GF), and said determin- 
ing degrees of menibersfup of the ver^ pattern to said second 
fuzzy sets "SmalT and "Big" provides for determining a third and fourth membership functions depending on said 
global features (GF). 

13. Method according to claim 12, characterized in that said groups of predef ined weight factors (cij) cxxnpnse groups 
of predefined horizontal weight factors and groups of predefined vertical weight factors, each fuzzy rule of said first 
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set being associated with a respective one of said groups of predefined horizontal weight factors, and each fuzzy 
rule of said second set being associated with a respects 

14. Method according to claim 13. characterized in that said weight factors (H1 -H5.V1-V5) comprise horizontal weight 
factors (H1-H5) and vertical weight factors (V1-V5), said horizontal weight factors (H1-H5) being determined by 
making a weighted average of the predefined horizontal weight factors with weight coefficients being formed by the 
activation degrees (a1-a9) of the fuzzy rules of the first set and said vertical weight factors (V1-V5) being deter- 
mined by mating a weighted average of the predefined vertical weight factors with weight coefficients formed by 
the activation degrees (pi -09) of the fuzzy rules of the second set 

15. Method according to daim 14, characterized in that the value of the pixel (Px) under examination is modffied by 
applying the horizontal weight factors (H1-H5) to the values of the pixels (Pa-Pd,Px) in the horizontal processing 
window (HPW) and applying the vertical weight factors (V1 -V5) to the values of the pixels (Pe-Ph,Px) in the vertical 
processing window (VPVV). 

16. Post-processing device for reducing artifacts in block-coded digital images, characterized by comprising: 

- first means (1) supplied with an input image (In) for estimating global features (MOOT) of an image block (IB) 
under examination, said global features providing information on an average content of image edges along the 
rtorizontal and vertical directions of said image block (IB); 

- second means (7,8.11,12,0) supplied with said input image (In) tor estimating local features (VS1-VS5.HS1- 
HS5) for each pixel (Px) of the image Hock (IB) under examination, said local features providing information on 
the content of image edges along the horizontal and vertical directions of an image area around said pixel (Px) ; 

- third means (9.0.10) supplied with said global features (MOOT) and said local features (VS1 -VS5.HS1 -HS5) 
for rrxxfifying the value of said pixel (Px) according to both sad global features (MOUT) and said local features 
(VS1-VS5.HS1-HS5). 

17. Device according to daim 16. characterized in that said first means (1) comprises means 0.4) for evaluating hori- 
zontal and vertical Sobel operator outputs (Hsob, Vsob) of horizontal and vertical SobeJ operators appSed to pixels 
of an Image sub-btock (ISB) internal to said image bfock (IB) under examination, and accumulator means (5) for 
accumulating the horizontal Sobel operator outputs (Hsob) and the vertical Sobel operator outputs (Vsob) for each 
pixel of the image sub-block (ISB). 

18. Device according to claim 17, characterized in that said second means (7,8.1 1,1 2.D) comprises fourth means (7.8) 
for evaluating vertical Sobel operator outputs (VS1-VS5) of vertical Sobel operators (Vsob) applied to said pixel 
(Px) and to vertically-neighboring pixels (Pe.Pf.Pg.Ph) preceding and following the pixel (Px) in the vertical direc- 
tion, and fifth means (11.12.D) for evaluating horizontal Sobel operator outputs (HS1-HS5) of horizontal Sobel 
operators (Hsob) appfied to said pixel (Px) and to rtcfizontafly-neigriboring pixels (Pa.Pb.Pc.Pd) preceding and fol- 
lowing the pixel (Px) in the horizontal direction. 

19. Device accorrjng to claim 18. characterized in that said third means (9,0,10) comprises vertical filter means (9,D) 
supplied with said vertical Sobel operator outputs (VS1 -VS5) and said global features (MOUT) for calculating a ver- 
tically-filtered value (BQ of said pixel (Px) depending on said global features (MOUT). said vertical Sobel operator 
outputs (VS1-VS5) and the values of said vertk^yHraighboring pixels (Pe.Pf.Pg.Ph). and horizontal fitter means 
(10) supplied with said gfobal features (MOOT), said horizontal Sobel operator outputs (HS1 -HS5) and said vert- 
caBy-fiitered value of said pixel (Px) for calculating a rwrizontalry-firtered value of said pixel (Px) depending on 
said global features (MOOT), said horizontal Sobel operator outputs (HS1-HS5) and the vertcaOy-ntered values 
(fafoftuEfcEd) of said pixel (Px) and said horizontally-ne^hboring pixels (Pa.Pb.Pc.Pd). 

2a Device according to claim 1 9, characterized in that said vertical f ater means (9. D) comprises first fuzzy computation 
means (13) supplied with said vertical Sobel operator outputs (VS1-VS5) and said global features (MOOT) for eval- 
uating degrees tfcoinctfenc*^ 

defined patterns of vertical Sobel operator outputs, said degrees of coincidence (01-09) depending on said global 
features (MOUT), a look-up table of predefined vertical filter coefficients (F1-F9) each one associated to at least 
one respective predefined pattern of vertical SobeJ operator outputs averts 

(14) supplied with said predefined vertical filter coefficients for generating a set of vertical fitter coefficients (VI -V5) 
which area weighted average of the predefined vertical fitter coefficients with weight coefficients formed by said 
degrees of coincidence (01-09), and a vertical fater (15) supplied with said vertical fitter coefficients (V1-V5) and 
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tto valines of the pixel (Px) and the vertx^^ 
vertcaly-fittered value of the pixel (Px). 

21. Device according to claim 20, characterized in that said horizontal fitter means (10) comprises second fuzzy com- 
putation means (13) supplied with said horizontal Sobel operator outputs (HS1-HS5) and said global features 
(MOUT) evaluating degrees of coincidence (al-o9) of the pattern of horizontal Sobel operator outputs (HS1- 
HS5) with a set of predefined patterns of horizontal Sobel operator outputs, said degrees of coincidence (a1 -a9) 
depending on said global features (MOUT), a look-up table of predefined horizontal fitter coefficients (Fi -F9) each 
one associated to at lead one respective predefined pattern of horizontal Sobel operator outputs, a horizontal Wter 
coefficients composition means (14) supplied with said predefined horizontal ffter coefficients (F1-F9) for general- 
ing a set of horizontal titer coefficients (H1 -H5) which are a weighted average of the predefined horizontal titer 
coefficients with weight coefficients formed by said degrees of coincidence (a1 «a9). and a horizontal inter (15) sup- 
pied with said horizontal filter coefficients (H1-H5) and the vertically-filtered values (i^.Pb.Px. Pc.Pd) of the pixel 
(Px) and the horizontally^ghboring pixels (Pa.Pb.Pc.Pd) lor providing at an ouput (FOOT) said horizontally-fil- 
tered value (Out) of the pixel (Px). 
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A Generalized Block-Edge 
Impairment Metric for Video Coding 

R R. Wu and M. Yuen 



Abstract — A new generalized block-edge impairment metric 
(GBIM) is presented in this paper as a quantitative distortion 
measure for blocking artifacts in digital video and Image coding. 
This distortion measure does not require the original image 
sequence as a comparative reference, and is found to be consistent 
with subjective evaluation. 

index Terms — Blocking artifacts, quantitative impairment met- 
ric, video coding. 



I. Introduction 

FOR bit rates ranging from 64 kbps (ITU FU61) to as 
high as 10-12 Mbps (MPEG-2) [I], we have shown 
in [2] that blocking effects, and its propagation through 
reconstructed video sequences, are the most significant of 
all coding artifacts. The blocking effect is also a source of 
a number of other types of reconstruction artifacts, such as 
stationary area granular noise [3]. It is well known that many 
forms of quantitative quality metrics, or distortion measures, 
used in image and video coding research, such as mean squared 
error (MSE), peak signal-to-noise ratio (PSNR), and mean 
absolute error (MAE) [1], are ineffective in quantifying the 
visibility of reconstruction artifacts, and, therefore, are ill 
suited for evaluating image and video compression techniques 
and products. The reason for this relates to the fact that these 
measures do not necessarily reflect our visual perception of 
the coding distortions and artifacts found in reconstructed 
sequences. Consequently, there is a poor correlation between 
these metrics and subjective assessment [4]. 

Recently, Karunasekera and Kingsbury introduced a new 
distortion measure for blocking (edge) artifacts in compressed 
images based on human visual sensitivity [4]. This quantitative 
distortion measure requires both the original and reconstructed 
images to form an error image, which is then used as an input 
into their visual model. This approach is similar to most of the 
other existing quantitative distortion measures [5], [6]. In the 
absence of the original image, the above distortion measures 
cannot be used to evaluate the quality of a reconstructed image 
or the presence of coding artifacts, such as the blocking effect. 
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Fig, I. The weighting function for the generalized BIM. 



In [2], the block-edge impairment metric (BIM) was intro- 
duced to evaluate the visual significance of block-edge artifacts 
in a given image. The BIM was based on a formulation 
of constraint sets applied successfully in the postfiltering of 
reconstructed video images using projections onto convex sets 
(POCS) algorithms [7]. Whilst achieving results consistent 
with subjective evaluations, this metric was characterized by 
the fact that it did not require the original image to produce a 
measure of the distortion in a reconstructed image. 

To take into account the luminance masking effects in 
extreme bright as well as extreme dark areas in a reconstructed 
image, a more general formulation of the BIM is presented in 
Section II of this contribution, parameters of which can be 
adjusted to correspond with the visibility of noise modeled 
in [8]. Section III will present experimental results using this 
generalized BIM as a distortion measure compared with PSNR. 

II. A Generalized Block-Edge Impairment Metric 

In [8], Girod suggested that distortions are most noticeable 
where the luminance value is between 70 and 90 (centered 
approximately on 81) in 8-b grey-scale images. To accommo- 
date this observation, and to maintain the simplicity of the 
weighting function in the BIM, a generalized BIM (GBIM) is 
formulated as follows. 

Given an image f = {fcifca • • tjvj, where Uj is the j'th 
column of the image array and N c is the width of the image, we 
define the interpixel difference between each of the horizontal 
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